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Abstract: The Internet has brought almost limitless possibilities for the 
promotion of services and products and thereby caused a significant change 
in the world. It also allowed beginners to get more important information 
about their future jobs. Based on this idea, the research and writing of this 
work began. In this paper, we investigate how much IT companies' websites 
provide information on company products, used technology and relationships 
with employees. The paper presents a multicriteria model for evaluating IT 
companies' websites from the point of view of young IT experts. 
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Castcemax: Humepnem je ðoneo ckopo neoepanusene moeyħnocmu 3a 
npomoyujy ycryea u npouz60dd u mume u3a36a0 3HAYAajHE NpoMeHe y ceemy. 
Taxohe je omoeyħuo u noyemnuųyuma ða Oobujy 3HauajHe unqopmayuje o 
Houxoeum Oydyhum nocioeguma. 3acHoBaHoO Ha oeoj uðeju, sanouenro je 
UCMPAHCUBATE u Nucatee o602 pada. Y o6oom pady, ucmpagcyje ce KoUKO 
ee6 cajmoeu IT komnanuja npyocajy uxdopmayuje o KomnanujcKum 
npouseoðuma, Kopuuhenoj mexnonoeuju u OdHOCUMAa ca 3anocaeHuma. Pad 
npedcma6sa BulueKpumepujymMcKuU Model 3a ebanyayujy ee6 cajmoea IT 
Komnanuja uz nepcnexmue6e miaodux IT excnepama. 

Kayune peau: Ananusa 6e6 cajma, Single valued neutrosophic set, 
6uuekpumepujymcKa eeallyayuja. 


1. INTRODUCTION 


The website can help you in gaining information about the company, 
products and services. However, the existence of a website does not 
automatically provide a competitive advantage. So, there is one important 
question: How much website actually meets the requirements of its users and 
how to measure its quality? 


In the literature, numerous studies have been devoted to the evaluation of web 
site quality. Boyd Collins developed the first formal approach to the 
evaluation of websites in late 1995. His model, intended for librarians, has 
been based on six criteria, developed by combining evaluation criteria for 
printed media, and considering what was relevant for websites (Merwe & 
Bekker, 2003). The model contains the following criteria: contents, authority, 
organizations, searchability, graphic design and innovation use. 


Studies that are intended for the identification of key evaluation criteria, and / 
or their significances, are still actual. For example, Dumitrache (2010) gives 
an overview of criteria used for evaluation of e-Commerce sites in Romania, 
during the period 2006 and 2009. As very important criteria it defines 
Response Time, Navigability, Personalization, Tele-presence and Security. 
Davidaviciene and Tolvaisas (2011) identify the list of criterions for quality 
evaluation of e-Commerce website. They also provide a comprehensive 
overview of the criteria that have been recently proposed by different authors. 
In accordance with (Davidaviciene & Tolvaisas, 2011) criteria: Easy to use, 
Navigation, Security assurance, Help (real time) and Design have been 
discussed by numerous authors, such as (Loiacono et al., 2007; Parasuraman 
et al., 2007; Cao et al., 2005). 
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Compared to different types of e-commerce, the IT companies have its own 
peculiarities. Therefore, for the evaluation of IT companies’ websites, we 
must use the appropriate set of criteria, and their significances. 


The Internet has also brought significant opportunities for many less known 
IT companies. 


In many countries, the development IT industry is often mentioned as one of 
the priority directions of development. A similar situation exists in Serbia, 
which has a number of attractive but also almost unknown IT companies. 


After selecting certain IT companies, we probably want to learn more about 
them. Here, some questions arise: How much website of these IT companies 
provide the necessary information? To what extent IT companies use the 
benefits that the Internet provides? What information is provided to interested 
young IT experts? 


The answer to the above questions we get with measuring the quality of 
websites of some IT companies that are located in Serbia. 


Therefore, the rest of the paper is organized as follows: In the second section 
of the paper, some basic definitions related to the SVNS are given. In the 
third section of the paper, is proposed the criteria for evaluating websites 
from the standpoint of the new it professionals. In the fourth section is 
proposed the procedure for evaluating websites based on the use of adapted 
SWARA method and SVNS. In the fifth section, the proposed model has 
been applied to the evaluation of five IT companies that are in Serbia. At the 
end of the paper, the conclusions are presented. 


2. SINGLE VALUED NEUTROSOPHIC SET AND NUMBERS 
Definition. Single valued neutrosophic set (SVNS). Let X be the universe of 


discourse (Wang et al., 2010). The SVNS A over X is an object having the 
form 


A={x<T,(),1,(%), F,(X) >| xe X} (1) 
where Ta(x), Ia(x) and Fa(x) are the truth-membership function, the 


intermediacy-membership function and the falsity-membership function, 


respectively, T,,1,,F,:X [-0,.*Jand 95440) +L) + FQ) S 3 
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Definition. Single valued neutrosophic number. For the SVNS A in X the 
triple <t,,i,, f, > is called the single valued neutrosophic number (SVNN) 
(Smarandache, 1999). 


Definition. Basic operations on SVNNs. Letx, =<t,,i,,f,> and 
x, =<t,,1,, f, > be two SVNNs, then additive and multiplication operations 
are defined as follows (Smarandache, 1998): 
X +x, =< +t, hh hi fh, ee (2) 
XX =< hhh th =i fith fh >, (3) 


Definition. Scalar multiplication. Let x=<t,,i,,f,> be a SVNN and 
A >0, then scalar multiplication is defined as follows (Smarandache, 1998): 


Ax, =<1- 0-6) >, (4) 


Definition. Power. Let x =<t,,i,, f, > bea SVNN and å >0, then power is 
defined as follows: 


xf =< i -A-S > 5) 


Definition. Score function. Let x =<t,,i,, f, > be a SVNN, then the score 
function s, of x can be as follows (Smarandache, 1998): 


Sa =(l1+t, —2i, — f,)/2 (6) 


where s, €[-1,1]. 
Definition. Single Valued Neutrosophic Weighted Average Operator. Let 
A, =<t,,i,,f, > be a collection of SVNSs and W=(w,,w,,...,w,) is an 


associated weighting vector. Then, the Single Valued Neutrosophic Weighted 
Average (SVNWA) operator of Aj is as follows (Sahin, 2014): 


SVNWA(A,, A,,..-,A,) = >, WjA; 


j=l 


, (7) 
-h1- fja- feon fje" | 
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where wj is the element j of the weighting vector, w, <[0,1] and >’_,w,=1. 





3. THE CRITERIA FOR EVALUATING WEBSITES FROM THE 
STANDPOINT OF THE NEW IT PROFESSIONALS 


In the literature, numerous studies have been devoted to the evaluation of 
website quality. As a result, a number of criteria have been proposed for the 
evaluation of websites. 


However, it should be taken into account that these studies were designed for 
the evaluation of different types of websites. 


Therefore, in this case, a set of criteria that realistically reflects the goals of 
the IT beginners is selected. Due to the use of SVNN, a set of the evaluation 
criteria that containing a smaller number of criteria more complex was 
selected, as follows: 
— Cı- About Us, 
— Cə- Products and Services, 
— C- Technologies, 
— C4- Carrier and benefits, 
where: 
— the criterion C, includes general information about a company that can 
serve to assess its relevance in the IT industry; 
— the criterion C2 more precisely defines the scope of the company's 
business; and 
— criteria C3 and C4 indicates IT technologies that used in a company, as 
well as the possibility of advancement, which can be very important for 
new IT beginners. 


4. THE PROCEDURE FOR EVALUATING WEBSITES BASED ON 
THE USE OF ADAPTED SWARA METHOD AND SVNS 


In each multiple criteria evaluation process, the following three important 

activities could be identified: 

— determining the importance of the evaluation criteria, 

— evaluation, where the alternatives are evaluated in relation to the selected 
set of evaluation criteria, and 

— aggregation and ranking alternatives. 


In this approach an adaptation of the SWARA method is accepted for 


determining criteria weights. The SWARA method is proposed by KerSuliene 
et al. (2010), and this method can be considered as efficient and easy to use. 
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Therefore, this method is used to solve a number of decision-making 
problems. Unfortunately, the computational procedure of the SWARA 
method is based on the usage of an ordered list of evaluation criteria, 
presorted according to their expected significances, which can be a real 
limitation when it is necessary to collect the real attitudes of in advice 
unprepared respondents. Therefore, Stanujkic et al. (2017) proposed an 
extension of the SWARA method that does not require the use of a pre-sorted 
list of evaluation criteria, on which basis the weight of the criteria are 
determined in this approach. 


However, the use of a large number of criteria can lead to forming a more 
complex MCDM models that could be less practical for the use in the cases 
when researches are based on gathering real attitudes of in-advice unprepared 
respondents. Therefore, this approach is based on the use of a smaller number 
of criteria that are evaluated using SVNN. 


Finally, for the aggregation phase, a procedure based on the application of the 
SVNWA operator and the Score function is selected. 


5. A NUMERICAL ILLUSTRATION 


In order to explain the proposed approach in detail, below is considered an 
example of evaluation of the websites of five IT companies’. 


In the conducted research, the evaluation of the websites of the following IT 
companies was carried out: 

- Comtrade, available at: https://www.comtrade.com/; 

- Levi Nine, available at: https://www.levi9.com/; 

- NIRI IC, available at: www.niri-ic.com/; 

- AB Soft, available at: www.absoft.rs/; and 

- Informatika AD, available at: www.informatika.com/. 


It should be stated here that the aim of this article is not to promote any of the 
above listed companies, because of which the order of the alternatives in the 
presented example does not correspond to the order of the above companies. 


The responses obtained from the first of three considered respondents, and 
weights of criteria, obtained by using extended SWARA method, are 


encountered in Table 1. 


The attitudes obtained from the three examinees, as well as the appropriate 
weights and group criteria weights, are presented in Table 2 as well. 
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Table 1. The responses and weights of the criteria obtained from the first of 


three evaluated respondents 





























Criteria Sj kj qj Wj 
Cı | About Us 1 1 0.22 
Ca | Products and Services 0.90 | 1.10 | 0.91 0.20 
C3 | Technologies 1.20 | 0.80 | 1.14 0.25 
C, | Carrier and benefits 0.60 | 1.40 | 0.81 0.18 
Se Silt 1.00 
Table 2. The attitudes and weights obtained from the three examinees 
Ei 1% E3 
Sj Wj Sj Wj Sj Wj Wj 
Ci 0.19 0.05 0.19 | 0.14 
Cə | 1.20 | 0.24 | 1.80 | 0.27 | 1.20 | 0.23 | 0.25 
C3 | 1.15 | 0.28 | 0.80 | 0.23 | 1.10 | 0.26 | 0.25 
C4 | 1.05 | 0.29 | 1.50 | 0.45 | 1.20 | 0.32 | 0.36 


The following are the responses obtained from the three examinees regarding 

































































the evaluation of the websites. 


Table 3. The ratings obtained from the first of the three examinees 

























































































Cı C2 C3 C4 
Aı | <0.5,0,0.2> | <0.7,0,0> <0.7,0,0> <0.8,0,0> 
A2 | <0.7,0,0.5> <0.8,0,0> <0.9,0,0> <0.9,0,0> 
A3 | <0.15,0,0.2> | <0.3,0,0.15> | <0.2,0,0.3> | <0.1,0,0.4> 
A4 | <0.25,0,0.3> | <0.2,0,0> | <0.15, 0,0.2> | <0.05,0,0.3> 
As | <0.2,0,0.4> <0.4,0,0> | <0.1,0,0.5> | <0.05,0,0.2> 
Table 4. The ratings obtained from the second of the three examinees 
Ci C2 C3 C4 
Aı | <0.8, 0, 0.2> | <0.9, 0, 0> <0.8, 0, 0> <0.8, 0, 0> 
Az | <0.5, 0, 0.6> | <0.8, 0, 0> <0.6, 0, 0> <0.7, 0, 0> 
A3 | <0.1, 0, 0.8> | <0.3, 0, 0> <0.35, 0,0.8> | <0.2, 0, 0> 
A4 | <0.2, 0, 0.6> | <0.2, 0, 0> <0.3, 0, 0> <0.1, 0, 0> 
As | <0.6, 0, 0.3> | <0.1,0,0.9> | <0.2, 0, 0> <0.1, 0, 0> 
Table 5. The ratings obtained from the third of the three examinees 
Ci C2 C3 C4 
Aı| <1,0,0.1> <0.5, 0, 0> | <0.9, 0, 0.1> | <0.8,0.1,0.1> 
Az | <0.5,0.1,0.3> | <0.4, 0, 0> <1, 0, 0> <0.8,0.1,0.1> 
A3 | <0.4,0,0> | <0.4,0,0.1> | <0.4, 0.1, 1> | <0.4,0.3,0.2> 
Aq | <0.2,0.2,0.2> | <0.1, 0, 0> | <0.3,0.3,0.1> | <0.1, 0, 0.4> 
As | <0.5,0.2,0.1> | <0.1,0.2,0.1> | <0, 0.3, 0.5> | <0.1, 0, 0.4> 
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The group ratings and overall ratings, shown in Table 6 and Table 7, are 
obtained by using SVNWA operator, or more precisely by using Eq. (7). 


Table 6. The group ratings 



































Ci C2 C3 C4 
IA; | <1,0,0.16> | <0.75, 0, 0> <0.82, 0, 0> <0.8, 0, 0> 
A> |<0.58,0,0.45> | <0.71, 0, 0> <1, 0, 0> <0.82, 0, 0> 
A3 |<0.23,0,0.03> |<0.34,0,0.01> | <0.32,0,0.62> |<0.24,0,0.02> 
A4 \<0.22,0,0.33> | <0.17,0,0> | <0.25,0,0.01> |<0.08,0,0.02> 
IAs |<0.46,0,0.23> \<0.21,0,0.02> | <0.1,0,0.03> |<0.08,0,0.02> 








The ranking order of considered alternatives, obtained based on the values of 
the score function, calculated by using Eq. (6), is also presented in Table 7. 


Table 7. The overall ratings 



































Overall ratings Si Rank 
A <1, 0, 0> 0.9992 2 
Ad <1, 0, 0> 0.9993 1 
A3 | <0.29, 0,0.04> | 0.6207 3 
A4 | <0.17, 0,0 .01> | 0.5807 4 
As | <0.19, 0,0.03> | 0.5767 5 





As it can be concluded on the basis of the data presented in Table 7, the most 
promising company from the perspective of an IT beginner is the company 
labelled as Az, which is somewhat more promising than a company 
designated as Aj. 


6. CONCLUSION 


This paper proposes a simple but also effective framework, which can be 
used for measuring the quality of IT companies’ websites from the 
perspective of IT beginners. 


The proposed procedure for evaluating websites based on the use of adapted 
SWARA method and SVNS has been successfully applied to the evaluation 
of five IT companies. 


Using this procedure, the managers of IT companies, can evaluate their and 
competing websites and compare them. This would continually influence the 
improvement of the quality of the websites and thus make it easier for 
youngsters to get to the desired information about the IT company. 
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